Sensor systems incorporating a plurality of sensors (multi-sensor systems) are widely used for a variety of military applications including ocean surveillance, air-to-air and surface-to-air defense (e.g., self-guided munitions), battlefield intelligence, surveillance and target detection (classification), and strategic warning and defense. Also, multi-sensor systems are used for a plurality of civilian applications including condition-based maintenance, robotics, automotive safety, remote sensing, weather forecasting, medical diagnoses, and environmental monitoring (e.g., weather forecasting).
To obtain the full advantage of a multi-sensor system, an efficient data fusion method (or architecture) may be selected to optimally combine the received data from the multiple sensors to generate a decision output. For military applications (especially target recognition), a sensor-level fusion process is widely used wherein data received by each individual sensor is fully processed at each sensor before being output to a system data fusion processor that generates a decision output (e.g., “validated target” or “no desired target encountered”) using at least one predetermined multi-sensor algorithm. The data (signal) processing performed at each sensor may include a plurality of processing techniques to obtain desired system outputs (target reporting data) such as feature extraction, and target classification, identification, and tracking. The processing techniques may include time-domain, frequency-domain, multi-image pixel image processing techniques, and/or other techniques to obtain the desired target reporting data.
It is advantageous to detect or identify image elements or targets as far away as possible. For example, in battle situations, candidate or potential targets should be detected early, increasing the likelihood of an early detection of a target or other object. For a simple background scene such as a blue sky, a target may be recognized from a relatively long range distance. However, for some high clutter situations such as mountains and cities, the detection range is severely reduced. Moreover, such clutter situations are often complicated to process. For example, the background may be mixed with different clutter types and groups. Also the background clutter may be non-stationary. In these types of situations, the traditional constant false alarm ratio (CFAR) detection technique often fails.
Spatio-temporal fusion for target classification has been discussed in the art. The fusion is conducted in the likelihood function reading domain. In general, the likelihood functions (pdfs) are obtained from training data based on single sensor and single frame measurements. Therefore, fusion is conducted using the likelihood readings of the features extracted from measurements of single sensor and frame, only one set of likelihood functions needs to be stored for a single sensor and frame, no matter how many sensors and frames are used for fusion. On the other hand, if the detection process uses thresholding technique instead of likelihood functions, the features values can be directly fused from different sensors and time frames in the feature domain for target detection.
Spatial fusion is defined as the fusion between different sensors, and temporal fusion is defined as the temporal integration across different time frames within a single sensor. Accordingly, it is desirable to develop and compare different spatial fusion and temporal integration (fusion) strategies, including pre-detection integration (such as additive, multiplicative, MAX, and MIN fusions), as well as the traditional post-detection integration (the persistency test). The pre-detection integration is preferably conducted by fusing the feature values from different time frames before the thresholding process (the detection process), while the post-detection integration is preferably conducted after the thresholding process.